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ABSTRACT 

This discussion traces the legislative^ fiscal, and 
programmatic history of Project Follow Through and analyzes the many 
problems and controversies that have itirrounded it. It is argued that 
problems have been largely_a_ result of the attempt to cpmbirie 
research and development (R 6 D) and social act ion_ service programs. 
Lessons learned as a resultof experience with Follow Through suggest 
that (1) large educational R & D programs should not be abandpnedi 

(2) combining service delivery and R&D programs is very difficult; 

(3) ambiguities in authorizing legislation should be resolved; (4) a 
need exists for programmatically useful information on effective 
compensatory education approaches ; (5| program effectiveness is only 
slightly related to cost^ managerial properties primarily account for 
program effectiveness, structured programs are more effective with 
disadvantaged children than unstructured programs , and new R&D 
activitieis should stress complete projects and models^ and (6) for 
the purpose of assessing program effectiveness, randomized or 
quasi-experimental designs are preferred , programs assessed should be 
clearly different^ replications of all models studied should be made 
in suf f icient hiimber , and program implemehtatibh should be considered 
ail essential factor. (RH) 
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WHAT HAVE WE LEARNED FROM FOttOW THROUGH? 
1^ Implicatiohs for Future R&D Programs 

rsi 

Although the historical background of Follow Through is well-known 
to those who have followed the program and are today involved in planning 
its future^ it is nevertheless useful to trace the highlights of this back- 
ground so that we are all on common ground^ and ariy newcomers to 
this remarkable social program arena will have a basis for understanding 
its fascinating and distressing characteristics. 

Follow Through was one of several education programs born out of 
the War on Poverty during the middle and late 60s. Though it is hot 
widely known, all of the OEO programs came out of a larger plan which 
attempted to lay but the causes and cures of poverty in American society 
bh a systematic basis (4). The OEO planners reasoned that poverty should 
be attacked on four major fronts: 

Macro economic policy to maintain high employment; 
A negative income tax program to provide income for 
those who because of age, infirmity, or other basic 
reasons could hot fihd ehnplbymeht; 
Y^Ba^ " Manpower training programs for adblescehts ahd adults 

whose lack of skills and credentials prevented them 
from entering the labor market or remaining there, and; 
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Compensatdry edlicatidri programs for the purpose 
of preventing individuals from deprived backgrounds 
from reaching adulthood without the basic educational 
and coping skills necessary for employment. 
Macro economic policy was^ of course, subject to many influences 
and requirements, and the fiscal and political forces never congealed 
to support full negative income tax legislation. (A major experinient 
in this area was mounted, however. ) So, apart frbnn the relabeling of 
some existing programs, the new thriists in the War on Poverty consisted 
of a variety of new programs in the manpower training and cdmpehsatbry 
education areas, as well as in health and community action. 

The centerpiece of the cbmpehsatbry education initiative was Head 
Start. At the time this prdgrarti was conceived and put together; 
psychologists, educators, and others were facing up fdf the first time tb 
the already known and well documented facts that: 

Disadvantaged children arrived at the first grade with 
educatidnal deficits which were clearly measurable 
at that time. 

The diff'^rences in intellectual proficiency and basic 

skills between these disadvantaged children and 

their middle class peers widened rather than narrowed 

as the children proceeded thrbugh schbbl. 

By the time children reached 7th or 8th grade, the gap 

seemed irremediably wide, and resulted in disadvantaged 
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children continually experiencing the psychblbgicaily 
devasting esxperihce of failurei 

Such children were much more likely to drop out of 
school and drift into a life of drugs, unemployment, 
crirne, and welfare. 
- Fronn all this it could be confidently predicted that 
the sartie pattern of disadvahtagement and failure 
would reproduce itself in the next generation. 
Thus, it was reasoned that if a program like Head Start could be 
effective, it could narrow or perhaps even close this gap in the preschool 
years ^ and thereby set the remediated children on an altogether different 
life course. 

Even the most optimistic protagonists of the Head Start strategy, while 
they over estinnated what was known about compensatory education and 
how much change such progranns could bring about, knew tiiat once 
children left the more enriched environment of the Head Start programs 
and returned to their disadvantaged homes^ any benefits which might have 
been achieved would either level off or recede. It was at this point in the 
reasoning and the programmatic history tliat the idea of a Follow Through 
program was conceived to provide continuing reinforcement and enrichment 
for tile Head Start graduates during the early elementary grades. 

Like Head Start, Follow Through was not proposed as a small scale 
R&D or pilot demonstration program. The intention was to begin at ah 
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appreciable level of program activity — more than $100 millibh — 
and expand in subsequent years to cover the full target population. At that 
time, Head Start and Follow Through were not freestanding^ separately 
authorized programs with their own separate appropriations. They were 
part of the larger Community Action Program and appropriation 
within OEO. When the Congress acted on the OEO budget in 1968, it did 
not specifically say no to the proposal for Follow Through but it provided 
only a modest addition to the total Community Action Program budget — 
far less than the amount requested to inaugurate the Follow Through 
program. OEO had the task of distributing the budget, and understandably 
chose not to cut other programs in order to allow the new Follow Through 
program to begin at its fully requested level. The result was that its 
initial apprbpriatibh was $15 million. 

When this decision was made the original administrators of Follow 
Through made a wise, indeed, brilliant decision. They said, in effect, 
since we do not have enough money to launch a service program at any 
meaningful level, i. e. , one that could treat any significant portion of the 
target population, then let us convert Follow Through to an R&D program 
and use it to find out what approaches are effective in the field of 
compensatory education. 

Unfortunately, the full implications of this decision and the fundamental 
differences between R&D programs on the one hand and ideologically based 
social action programs on the other were not understood, and the subsequent 
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history of Follow Through hais largely been aii acrimbnibus and unresolved 
series of arguments among the major stakeholders in the program — 
program admiriistf ators, planners and evaluators, the Congress, and the 
program constituents — over what the program should be and how it 
should operate. 

Vigorous efforts on the part of the prdgfam managers, support from 
their constituents, and sympathetic reception in the Congress resulted iii 
raising the original budget level to a high of $59 million — far more thaii 
is needed for the most elaborate R&D program, but far less than would be 
needed to support a service program at any meaningful level. 

The R&D thrust of the program took the form of attempting to mount a 
planned variation experiment in which 17 different model programs would 
be studied and compafed. Unfortunately, this effort was poorly conceived 
and executed. The 17 models were not systematically develbped of 
selected, with the result that many were only marginally different from one 
another. Also, in many cases the model existed as little more than a few 
sketchy ideas arid took years to bring to the point where implementation 
could begin, in many cases implemeritatibh of the model in its various 
sites was neither monitored nor enforced^ with the result that the design 
heeded tb study the operations and outcomes of the different models^ 
namely their full replicatibn in several different sites, was achieved only 
for a few of the models. 

It is disappointing, though no one should have been surprised, that after 
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the expenditure of several hundred million dollars dii prbgrahi operations; 
arriari embarf asihgly large amount (in excess of $50 million) on a major 
longitudinal evaluation, the principal finding was that only one or two 
of the models could be said to bie producing appreciable attitudihal and 
achievement gains in the Follow Through children (1- 2). (However, 
reflecting the other side of the inter-site variation coin, a substantial 
number of individual Follow Through projects have presented solid 
evidence of effectiveness and have been approved for dissemination by 
FD's Joint Dissemination Review Panel (3). See below.) 

Frbna 1975 through 1978, successive administrations attempted to 
scale back or phase but the Follbw Through program, arguing that its 
service activities were largely redundant with Title I and should be 
folded into that program, and that the planned variatibh experiment, 
such as it was, had run its course, in every case^ Congress which has 
been quite suppbrtive of Follow Through, rejected the administration's 
proposals to in any way reduce the prbgram. ( The budget has been 
recently i-educed to $44 million.) Congress' defense bf the program 
has been based not only on its popularity and its appeal as an effort 
to help poor kids, but also on strong opposition to what it felt was the 
Nixon administration's unjustified efforts to dismantle the anti-poverty 
prbgrams. 

The Lessons of the Follow Throu^ Exper4e^ 

The lessbns from the Follow Through experience are painful, but 
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clearly we are obliged to learn from it. There is a great deal to be 
learned from the Follow Through experience, both at the level of how 
large government R&D programs should and should not be ran, as well 
as at the more micro level of how NIE should spend the limited Follow 
Through RfcD funds it is responsible for managing. 

i; Despite the manifold problems with Follow Through, we should 
hot conclude that large educational R&D programs, as a 
principal means of knowledge development and program 
improvement, should be abandoned. They can be made to work 
if we do not senselessly and mechanically rep»jat the programmatic 
methodical, and political errors of the past. 
2. A principal lesson to be learned from the Follow Through 

experience is the enormouis difficulty — some would say the 
clear impossibility — of trying to combine service delivery 
and R&cD progranns. They are very different enterprises and 
in some ways are fundamentally incompatible. Social programs 
thrive on ideological intensity, flamboyant public relations, 
unshakeabie commitments to programmatic objectives arid 
effectiveness, and skillful political manipulation. The purpose 
of R&D prbgrams^bh the other hand, is not to deliver services 
per se but to discover the most effective ways of doing that. 
Thus, their staffs must be technical rather than political or 
progf annmatic, and their attitudes skeptical and dispassionate 
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rather than committed or proselyting. The most fundamental 
difference between the two is that program managers and their 
supporters assume as established what researchers and 
evalaators are trying to determine. This means that in any 
future efforts to mount large scale experimental R&D programs, 
it is absolutely essential that the fundamental purpose of such 
programs be established at the outset, and that all the necessary 
steps are taken to ensure that the right kind of staff, procedures, 
objectives, outcome measures, limitations on time and funding 
horizons, etc. , are set forth and agreed upon. 

More than once I recall Congressmen asking staff members 
fr om the Follow Through program at Cbhgressibnal hearings 
such questions as, "How many children are you now serving?" 
The answer which should have been given but never was is^ 
"Mr. Chairman, the purpose of the Follow Through program 
is not to serve any particular number of children but to 
develop and assess effective ways for doing that. " Follow 
Through has been viewed by its own program staff- by most 
pieoplie in Congress, and by most of its constituents as mainly 
a service program^ and it has been examined and assessed 
on service, not R&D criteria. 

In a situation where the basic objectives of a program are 
either unclear or at issue, it is unfair to ask program oriented 
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adrriihistratbrs to make decisions and allocate resources for 
R&D purposes when they know they are going to be assessed 
by service program standards. Similarly, the research and 
evaluation memb<*rs of the program caniidt expect to have 
the program structured or operated in a way that will 
facilitate knowledge development. Therefore, the R&D 
part of the pf ogr am will cdritiriually face obstacles that will 
more than likely result in failure for its objectives, 
3. Since authorizing legislation for social programs is usually 
a cbmprbmise hammered but in an adversarial political 
context, more likely thari hot it will be fundamentally 
ambiguous on the program's service vs R&D brientatibn. 
It is therefore incumbent upon the Executive Branch agencies 
to aggressively and tenaciously pursue the matter of resolving 
these ambiguities^ unpleasant thbugh that task may be. The 
specific lesson to be extracted here is that Executive Branch 
agencies must work witih the Congress, must present a 
program plan which resolves ambiguities in the legislation- 
and must secure Congress' agreement or acquiescence to 
proceeding along a clear and specific line bf action. 
4. There is still a great need for programmatically useful 

information bh effective compensatory education approaches. 
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Despite the dissatisfaction and debate over the Westihghbuse 
evaluation of Head Start (5), no satisfactory evaluation of the 
effectiveness of that program has subsequently been done, 
with the result that we do hot know with any confidence whethe 
the program as a whole is effective^ or whether any of its 
sub-elements or components are achieving their objectives. 
As noted above, we have some useful knowledge coming out 
of the evaluation of Follow Through, but is is largely to the 
effect tiiat one or two approaches seem to work decently well 
and the rest either haven't been implemented^ or don't work, 
or both. The recent evaluations of Title 1 (2) are beginning 
to show some evidence of marginal effectiveness in that 
program, but both the magnitude and extent of gains are so 
small that at its current level of effectiveness the program 
can make only limited headway toward ameliorating the 
educational deficits of disadvantaged children. Thus, the 
decision to make anothrji: effort to organize and more 
effectively QT^loy the Follow Through R&D monies — as the 
Department of Education, the Follow Through program^ and 
NIE are how doing — is responsible and makes good sense. 
Despite all the justified larrieht over the failure in the past 
decade to develop a comprehensive body of knowledge about 
effective approaches to compensatory education ~ much less 
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tb solve the problem itself — some important lessons have 
emerged from bur research- evaluatibh, and programmatic 
experience with the several major compensatory educatibh 
programs. Among them are the following: 
a. In the increasingly critical financial circumstances 

being faced by mbst schbbls, and with the school bond. 
Proposition 13^ arid iriflatibri forces of the future lobkihg 
even grimmer, schools are simply not gdirig tb be in 
the market for compensatory education programs which 
require them to add the order of $800 on top of their 
present escalating per pupil expenses. Thus, even if 
the Follow Through models were effective, they have 
ho place to go; What is more important is that there is 
accumulating evidence that they heed-not be that costly 
to be effective. There is a grbwihg bbdy of evidence 
that indicates that program effectiveness is related 
very little to cost. One of the most compellirig but 
not widely known elements in this accumulating body of 
evidence is the cbllectibh of programs (3) approved by 
the iDepartment of Educatibri's Jbiht Dissemination 
Review Panel (JDRPj. These programs are examined 
rigorously for evidence of effectiveness befbre being 
apprbved for dissemination by BB programs. The 
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rate of approval is about 60% of submis sibhs , and 
' informal analysis of the compensatory education 
programs in this collection indicated they range in 
per pupil cost all the way from • 50 to over $2, 000. 
Most of tiaem are far below the $800 Follow Through 
figure, and there seems to be little relationship 
between their effectiveness and their cost. What this 
means specifically for the NIE R&cD program is that 
it should ensure that whatever models are developed are 
within the fiscal reach of public schools; otherwise, 
they will never be adopted or implemented. But more 
importantly, the evidence suggests that making such 
a limitation will not rule but quality or effectiveness i 
Ah important substantive lesson strongly suggested by 
the evaluation evidence from Follow Through, Title I, 
arid the many programs which have come before the 
JDRP is that mariagerial rather than content properties 
of a program primarily account for its effectiveness. 
Specifically, what this means is that programs which 
have such things as well structured training programs 
for the teachers^ objective selection mechanisms for 
the students, quantitative pre-post tests arid measurements 
of progress throughout the program, structured feed- 
back to students on their progress, well plariried arid 
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monitored lesson plans^ etc. , etc. - seerii to be more 
successful than programs which lack these properties- 
regardless of the content or curriculum they employ. 
Quite appropriately, the NIE planning docarhent takes 
note of this emerging finding arid recbmmerias that 
it be followed up and built upon . 

As a variation of this point, findings from the Follow 
Through and Title I evaluations and others, indicate 
that structured as opposed to loose, open, or self- 
guiding programs are more effective for disadvaritaged 
kids, especially in the early years; 
To be bperatibhally useful to classroom educators, 
the new R&D activities should stress complete projects 
and models rather than research on indivirlual educational 
variables. In any formulation of a tp|,at educational 
research program, there should of course be recognition 
of the need for research on the iridividual and interactive 
effects of all the major variables impinging on the 
learning situation. But the findings that emerge from 
these kirids of basic knowledge investigations are not 
what school administf atdfs arid teachers can apply in 
their classrooms. What they need is a clear set of 
elements and components that fit together into an 
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educational progracm or model that: 1) has been 
found to be effective when utilized in a broad array of 
similar classroom settings; and 2) they can with 
appropriate modifications put to work in their own 
classrooms with confidence that it will produce 
similar achievement outcomes. Contrary to the concerns 
of sdme^ there is no heed for this approach to be a 
Procrustean mode of program improvement. 
Moving from substance to matters of design, if what I have said 
above about the continuing need for information about effective 
cbrtlpehsatbry programs and the greater programmatic utility 
of project or model as opposed to individual variable kinds 
of knowledge is true, then it follows that at some point it 
nnakes sense to mount another fairly ambitious planned 
variation typie of study which would assess the effectiveness 
of alternative models and approaches to early childhbj>d 
compensatory education, if this is ever done — and there 
are many obstacles in the path to doing it — the Follow 
Through experience has taught us several clear lessons: 
a; Randomized or quasi experimental designs remain 

the best way to assess the effectiveness of cbmpehsatbry 
education programs, or for that matter anything else. 
Because of the time, cost, implementation difficulties, 
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ahd outcome ambiguities which have plagued many 
large quasi-esxperirhehtal evaluations, some researchers 
have been arguing that this basic methodology should be 
abandoned in favor of smaller^ more limited, and more 
informal approaches. This would be a mistake. The 
essence of program evaluation is the attribution of 
cause — the establishment that there has been a 
significant change in the prbgram population and that 
this change can be confidently attributed to the prbgram 
and hot some other cause or experience. The only 
satisfactory way we have of making such a determination 
is through experimental design type evaluations. The 
appropriate response to the difficultieis they ehcbuhter 
is not to retreat to a vastly more ambigious and there- 
fore mbre debatable kind of method, but to renew our 
efforts to correct the difficulties. 

One of the major difficulties in the Follow Thrbugh 
evaluation, as I noted above, was the failure at the 
outset tb establish clearly distinguishable models. 
Program managers yielded tb a variety of offers and 
pressures and settled on an overly large number bf 
models many of which were only marginally distin- 
guishable from one another. In future NlE assessments 
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of tiiis kind, care should be taken to select or develop 
a small numBer of clearly different approaches to 
cbmpensatbry education, approaches that are different 
fr orn otie another in theoretical base, in types of 
education expefiences offered, etc. It is only 
through a controlled examination of the outcomes 
of such clearly distinguishable approaches that we 
can really learn what works for whom in compensatory 
education. 

The Follow Through experience has also made clear 
the crucial importance of instituting a sufficient number 
of feplicatibhs of all the rnbdels under study. It is 
sophomoric to have to lecture ourselves bn such obvious 
considerations as: educational commilhities and cbntexts 
vary, and therefore what may be successful in one place 
may not be geheralizable; some sites will experience a 
variety of local difficulties and will fall out of the study 
entirely; one needs some reasonable number bf cases 
to analyze data statistically; etc. But the absence of 
sufficient replicatibns of the different models in the 
Follow Through evaluatibh was one bf the crippling 
shortcomings in that RfcD effort. 

The Follow Through experience also makes clear hbw 
essential and uncertain the matter of implementation is. 



17 



John W. Evans 
l/lb/81 

WHAT HAVE WE LEARNED FROM FOLLOW THROUGH? 
Implications for Future R&D Progfamis 
SUMMARY 

This paper traces the legislative, fiscal^ and programmatic history 
of the Follow Through program and analyzes the many problems and 
controversies that have surrounded it. In large part, these problems 
have resulted from its attempt to pursue the cbhflictinfi goals of being 
ah R&D program and a social action service progf am. 

Several lessons are extracted from the Follow Through experience: 

1. Large scale educational R&D programs should not be 
abandoned. By learning from the past, their substantial 
problems can be overcome and much can be learned frbrri 
them, 

2. Cbmbiriihg social action and R&D pfogfarhs is extraordinarily 
difficult, perhaps impossible. Their goals, procedures, 
outcome measures, and required types of staff are often 
fundamentally incompatible. 

3. Authorizing legislation is usually ambiguous on the key issue 
of whether the primary thrust of a program should be to 
deliver services or, through research^ to develop improved 
means of doing that. To avoid the problems encountered by 
Follow Through and other similar programs. Executive Branch 
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Agaiiij the Follow Through evaluation was plagued not 
only by insufficient replications of the various models 
but also by the fact that the models were poorly and 
variably installed across their various sites, with 
the result that instead of having seven replications of 
model A, we had two replications of model A and five 
other substantially different programs, or in some 
cases ho program at all, 

***** 

It has not been my purpose in assembling these lessons from the Follow 
Through experience to assign blame. It is easy in retrospect to see how 
things should have bieieh done differently, but vastly more difficult to have 
that wisdom at the time and be able to impliemeht it. 

But we can and must learn from these kinds of experiences. And if 
we learn what we should from 'Follow Through, it will greatly enhance 
our ability to rationally develop effective compensatory education programs, 
instead of waiting until they occasionally appear by chance. 
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agencies should work cloisely with the Congress to resolve 
these ambiguities. 
4. There is still a great heed for information on effective 

approaches to compensatory education. Therefoi-e, NiE's 
current effort to organize and more effectively employ 
the Follow Through R&D monies makes good sense. 
5. Despite all its problems, the Follow Through experience 

has provided us with a humbef of important lessons. Among 
them are the following: 

B,i The current level of per pupil expenditures in the 
Follow Through program — $800 — is too high to 
secure interest or adoption by financially hard pressed 
school districts. Lower cost programs should be 
sought, and there is an accumulating body of evidence 
which indicates that this will not be impossible to do, 
since effectiveness is not highly related to cost, 
b; Managerial father than curricular content properties 
of compensatory education progfams seem to be the 
major detefmihers of their effectiveness, 
c. St^idctured as opposed to loose or self guiding 

compensatory education programs appear to be more 
effective for disadvantaged kids, especially in the early 
yearsi 
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d. The develbprneht of cbrhplete educational projects 
or models is likely to be more useful to classroom 

educators than general scientific knowledge bh the 

interrelationships of educational variables. 
6. At some propitious time in the future, consideration should 
be given to mbuntihg another ambitious, planned variation 
type of study, if this is done^ the Follow Through experience 
provides us with a number of important lessons: 
a. Randomized of quasi experimental designs remain 

the best way to assess the effectiveness of compensatory 

education programs, 
bi The models in any such experiment must be developed 

or selectesd in a way to ensure that they are clearly 

distinguishable from one another, 
c; There must be an adequate number bi replications of 

each model. 

d. Steps must be taken to ensure that the models are in 
fact implemented across their various sites; 



22 



